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ABSTRACT 

This paper explores the use of K. G. Joreskog's 
(1970) congeneric modeling approach to reliability using censored 
quantitative variables. Two Monte Carlo studies were conducted. The 
first explored the robustness of Normal Theory Generalized 
Least-Squares (NTGLS) estimates for a single-factor congeneric model 
across several sample sizes (N=25,50, 100,400) , model loading sizes, 
and different levels of censoring (0%, 25%, 50%, 75%). The second 
study compared alternate estimation procedures for different levels 
of variable censoring (0%, 25%, 50%, 75%) based on a single-factor 
large loading congeneric model. NTGLS, asymptotically distribution 
free (ADF), and latent TOBIT estimators were compared as to their 
efficiency in estimating model parameters. Results of the first study 
indicate that convergence rate is inversely related to sample size 
and to the size of the model loadings. Censoring of variables 
produced the expected negative bias in estimates using NTGLS methods 
with the magnitude of the bias somewhat robust against sample size 
variation. Results from the second study indicate that the TOBIT 
estimates were robust with respect to model rejection and amount of 
parameter bias across various levels of variable censoring. Both 
NTGLS and ADF methods proved to be unsatisfactory both in model 
rejections and percent of bias. Thirteen graphs and three data tables 
conclude the document. (Author/TJH) 
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Abstract 



This paper explores the use of Joreskog's (1970) congeneric 
modeling approach to reliability using censored quantitative 
variables. Two Monte Carlo studies were conducted. The first 
study explored the robustness of Normal theory Generalized Least- 
squares estimates for a single factor congeneric model across a 
variety of sample sizes (N=25, 50, 100, 400), model loading sizes 
(large A = .9 and moderate \ = .6), and different levels of_ 
censoring (0%, 25%, 50% 75%). A second study compared alternate^ 
estimation procedures for different levels of variable censoring 
(0%, 25%, 50%, 75%) based on a single factor large loading 
congeneric model with a sample size of N=100. Both normal theory 
generalized least-squares (NTGLS) (JSreskog & Goldberger, 1972; 
Browne, 1974), asymptotically distribution free (ADF) (Browne, 
1982), and latent TOBIT estimators (Muthen, 1985; 1987a; in 
press) were compared as to their efficiency in estimating model 
parameters. 

Results from study one confirmed previous findings that 
convergence rate is inversely related to sample size, and to some 
degree size of the model loadings. Censoring of variables 
produced the expected negative bias in estimates using NTGLS 
methods with the magnitude of the bias somewhat robust against 
sample size variation. Results from study two indicated that the 
TOBIT estimates were robust with respect to model rejection and 
amount of parameter b?as across various levels of variable 
censoring. Both NTGLS and ADF methods proved to be unsatisfactory 
both in model rejections and percent of bias. 



Introduction 



Several procedures have been proposed for the estimation of 
interrater reliability, many of which have been based on an 
analysis of variance approach (Winer, 1962 ) . Though these 
procedures are used frequently they require acceptance of a 
number of assumptions (Saal, Downey & Lahey, 1980). One major 
assumption is that all the measures must have the same unit of 
measurement, sometimes referred to as "tau equivalency" (Lord & 
Novick, 1968). In response to this assumption, a covariance 
modeling procedure, based on Joreskog's (1970) general model for 
the analysis of covariance structures has been proposed (Werts, 
Linn & Joreskog, 1974; Van Der Kamp & Mellenbergh, 1976). 

While the covariance modeling procedure provides details 
about the assumptions surrounding the use of analysis of variance 
based reliability measures, it is not without some limitation. 
One major limitation is that the observed variables must be 
multivariate normally distributed for the appropriate estimation 
of parameters, if one is using the typical estimation procedures 
(e.g., maximum likelihood, or normal theory generalized least 
squares). This procedure would come under question if one 
attempted to analyze censored variables (variables that have a 
high concentration of cases at either end of the distribution). 

This paper explores the use of Jcreskog's (1970) congeneric 
modeling approach to reliability using censored quantitative 
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variables. Two Monte Carlo studies were conducted. The first 
study explored the robustness of Normal theory Generalized Least- 
squares (NTGLS) estimates for a single factor congeneric model 
across various sample sizes ( 25, 50, 100, 400 ), model loadings 
(large \ = .9 and moderate \ = .6), and levels of variable 
censoring (0%, 25%, 50% 75%?. A second study compared alternate 
estimation procedures for different levels of variable censoring 
(0%, 25%, 50%, 75%) for the congeneric model, using NTGLS 
(Joreskog & Goldberger, 1972; Browne, 1974), asymptotically 
distribution free methods (ADF) (Browne, 1982), and latent TOBIT 
estimators (Muth6n, 1985; 1987a; in press). These methods were 
then compared as to their efficiency in estimating model 
parameters. 



Congeneric Modeling Approach to Reliability 



The congeneric approach, based on covariance modeling 
(Joreskog, 1970) considers a rater as a test instrument, with a 
data matrix X(m x k), denoting rater scores for a randomly 
selected subject group (m), where m = t*ie number of observations 
(subjects) over (k) raters. Assuming the rows of X are 
independently distributed, each having a multivariate normal 
distribution, the -uugeneric model for a four-indicator 
reliability model may be written as: 
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e. 



= A \ + e g (i) 



where (X) and (9 g ) are parameters to be estimated for each (k) 
rater, with (U) representing the true component and (9 £ ) the 
error component. Assuming uncorrelated measurement errors, with 
variables that have a zero mean and a factor variance of 1.00, 
the variance-covariance matrix for this model may be defined as: 



* 2 1 + % 
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which may be rewritten as: 



S - A A» + o 



(2)3) 
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where 8 £ is an (k x k) diagonal matrix of error variar.ces, and A r 



is a vector of loadings on a single common factor ♦ Using this 
model, one is able to test the initial assumption that all 
measures have the same underlying true score (\) . For example, if 
one considers a situation where four independent assessments of 
patient symptoms are rated using a continuous symptom scale, one 
would estimate the A loading matrix and the error variances 0 £ . 
For convenience, as previously mentioned, the variance of the 
true score may be standardized « 1.00). This approach will 

provide two overidentif ying restrictions in the model (two 
degrees of freedom), allowing one to test the following position: 



If this null hypothesis is not rejected, individual rater 
reliability may be estimated as: 

A 2 



A 



H 



S = E 
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(3) 



with composite reliability being estimated as: 
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Subsets of the Congeneric Model 



A major assumption in the use of intraclass correlation and 
general izability theory procedures ( Cronbach, Ra jaratnam & 
Gleser „ 196 3) is that all measures have the same unit of 
measurement, sometimes referred to as being M tau~equivalent rr 
(Lord & Novick, 1968). If this assumption is not met, then the 
averaging of scores would not be meaningful, and the use o£ ANOVA 
based procedures would not be appropriate* Joreskog (1970) 
provides a procedure based on a subset of the congeneric model to 
test for tau-equivalence . This assumes that measures are 
equivalent if the regression weights are equal in A* In the 
tau-equivalence model, one would test: 

H o ? X l = X 2 = h s X 4 

with (%k(k + l)-(k + 1)) degrees of freedom. A total of (k + 1) 
parameters would then be estimated* If the tau-equivalency 
hypothesis is rejected, then ANOVA procedures should be rejected 
for estimating reliability and/or generalizability procedures 
(werts, Linn & Joreskog, 1974)* If the hypothesis is not rejected 
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the parameter estimates may be used to estimate individual 
reliability: 



A 



A 2 
X 



x + e 



(5) 



and composite reliability: 

2 

(k X ) 

P c = (6) 

A 2 

(k a ) + e e, 
e i 

It should also be noted that the reliabilities estimated 
from equation (5) may vary contingent upon differing error 
variances (9 £ ). Tho stability of error variances is assumed in 
the ANOVA procedures, but may be tested using another subset of 
the congeneric model, sometimes referred to as a "parallel model" 
CGulliksen, 1968; JSreskog, 1970). This model implies equality 
constraints on both the ^'s and the 9 £ 's, so one tests the 
following: 



vjltbr nag 



O 



with (%k(k + 1) - 2) degrees of freedom. A total of two 
parameters will be estimated in this specific model. If the 
parallel hypothesis* is rejected, Werts, et al . (1974) assert that 
the use of ANOVA based procedures would have underestimated the 
composite reliability. If the parallel hypothesis is not rejected 
on may estimate the reliabilities as follows: 



P, 



*2 
X 



*2 * 

x + e, 



(7) 



and composite reliability: 



* 2 
(k X ) 



2 * 
(k X ) + k e, 



(8) 



Tests of these hypotheses amount to tests of the validity of 
specific assumptions about the model, which are not provided in 
the ANGVA based intraclass reliability estimates (Werts, Linn & 
Joreskog, 1974). 

While the Joreskog (1970) procedure seems attractive for 
answering questions regarding measurement unit equivalency and 
stability of error variances in reliability models, the procedure 
has limitations of its own. 
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Congeneric Model Limitations 

The test of the aforementioned hypotheses are somewhat 

contingent upon the type of parameter estimation procedure used 

in constructing the measurement model* Typically, normal theory 

maximum likelihood (HTML) or generalized least-squares (NTGLS) 

estimation procedures have been recommended, simply because these 

procedures allow the estimation of standard errors and provide a 

2 

test statistic (likelihood ratio L ) for assessing the hypotheses 
(Joreskog, 1970). The adequacy of using NTML or NTGLS estimation 
procedures for estimating such a reliability model must of course 
be based on a number of conditions* First, sample size seems to 
be a crucial factor. Monte Carlo studies based on normal theory 
distribution sampling have indicated that the use of NTML and 
NTGLS estimation procedures on sample sizes of less than 50 
result in model convergence problems, while sample sizes of N=25 
result in serious improper solutions (Boomsma, 1982; 1983; 1985; 
Anderson & Gerbing, 1984). Typically, minimum sample size 
recommendations for unbiased estimates are usually set at N 1 
100. 

Secondly, distributions with aberrant skewness and kurtosis 
values result in higher than usual model rejections (Boomsma, 
1983 ) and poorer parameter estimates. It is with this second 
limitation that this paper will concentrate. 
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Censored Continuous Variables 



One source of aberrant skewness and kurtosis values in sample 
variables has been the result of analyzing sample truncated 
normal distribution variables . Hald ( 1949 i has called such 
variables "censored" variables, since the population from which 
the sample variable was drawn is considered normally distributed, 
and the sample being considered incomplete. More formally, a 
continuous censored variable may be defined as: 

if y* * c 1 
if c x < y* < c u 
if y* * c u , 

where c^ and c^ represent known lower and upper censoring 
constants, which Aay take on values -« to +». Censored variables 
may thus be considered variables that have limited variability 
with a large proportion of cases occurring at one or both end 
point of the scale. Huth6n (1985; 1987a) has shown that Increases 
in censoring have a substantial effect on the attenuation of the 
population correlation estimates (Figure 1), which has an effect 
on model estimates. 




Insert Figure 1 about here 
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Censored variables are common place in the measurement of 
extreme phenomena, such as in the case of measuring psychotic 
symptomatology. A major question then becomes, how robust is the 
use of normal theory estimation procedures, specifically NTGLS, 
for estimating congeneric reliability models with censored 
variables? While studies have shown that normal theory methods 
are fairly robust with regards to mild deviations (Boomsma, 1983; 
Muth6n & Kaplan, 1985) questions remain regarding the robustness 
of NTGLS to moderate and sever deviations . 



Monte Carlo Studies 

Study Design 1 : 

Table 1 gives an overview of the first study's design. In 
this study only NTGLS estimates were used, and were obtained by 
using the LISCOMP (Muth6n, 1987b) computer program on an IBM- 
4341 mainframe computer . A standardized population covariance 
matrix (TJ) was created for the four variable single factor 
congeneric model using equation (2b). Two separate single factor 
model population covariance matrices were created. The first 
matrix (based on a highly reliable model with large loadings 
( LLM ) ) was established using; Var(\) « 1.00 and A 1 = t.9, .9, .9, 
. 9 ] . The second ma *r ix ( based on a less reliable model using 
moderate loadings (MLM) ) was established using; Var(\) = 1.00 and 
= 1*6, .6, .6, .6]. Variances of the errors in measurement 
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(uniqueness) were defined for both models as 9 g = l - \ 2 . Using 
equation (3), it may be shown that rater reliability for the 
congeneric model may be reduced to \ 2 . Table 2 provides 
population values and corresponding standard errors. Each E 
matrix was then used to generate 100 sample covariance (S) 
matrices for each sample size condition (N=25, 50, 100, 400). 



Insert 


Table 


1 


about 


here 




Insert 


Figure 
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about 


here 




Insert 


Table 
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about 


here 



Study Design 2 : 

Table 3 provides details about the design of the second 
Monte Carlo study. In this study three different estimation 
procedures were considered. First, as previously discussed, NTGLS 
was estimated over the conditions. Second, in an attempt to 
reduce the possible inflation in L due to non-normality, 
Browne's (19S2) asymptotically distribution free (ADF) 
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general i zed least-squares estimator was considered* Final ly, 

Muth6n's (1985; 1987a) latent TOBIT estimator for censored 

variables was estimated to: (1) provide a better estimate of the 

correlation, and (2) deal with non-normality* All estimates were 

obtained on an IBM-4 341 mainframe computer using the LISCOMP 

(Muth6n, 1987 ) program* At this point in the research, only one 

sample size (N=100) and one single factor congeneric model ( \ = 

2 

.9, with 6 £ = 1 - \ , and VarCH) = 1*00) was considered for the 
comparison analysis (see table 2, and figure 2). Future studies 
will explore variations in sample size, loading size and various 
model (multi-factor) . 



Insert Table 3 about here 



Estimation Procedures 

Estimations will be represented by the general family of fit 
functions for analysis of covariance structures, as proposed by 
Browne ( 1984 ) * The most familiar estimation procedure currently 
ussol is the NTGLS procedure* It's fitting function is: 



F GLS = (s -<r) r w(s-a). 



(10) 
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where (s) and (a) refer to (y) based on a Pearsonian 
correlation/covariance, with a weight matrix B=E®£orS®S, 
where E = A * A 1 + and a = VectE] (Joreskog & Goldberger, 

1972). 

The second estimator is Browne's (1982) asymptotically 
distribution free ( ADF) generalized least-squares estimator. The 
fitting function for this estimator is: 

F ArvTr = ( s - o)' V" 1 ( s - o), (11) 



where: 



S ijkl ~ S ij S kl' 



with s^ and biased sample covariances, and S^.^ is the 
fourth order multivariate cumulant (Kendall & Stuart, 1977)* 

The third estimator to be used is Muth6n's (1985; 1987a) 

latent TOBIT estimator ♦ It has the same fitting function as the 

NTGLS with the exception that instead of (y) referring to a 

* 

sample based Pearsonian correlation/covariance , (y ) refers to a 
larent TOBIT correlation/covariance, based on the underlying 
assumption of individual univariate normality. The TOBIT fitting 
function is: 

F TOBIT = ( S * ~ a * 3 B ~ 1 ( S * " a * )# (12) 
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* * * 

where s and o refer to y TOBIT based 

correlation/covariances, with a weight .matrix of E* ® E* or S* © 

S* , where E* = A * A' + 0 £ , and o* = Vec[E] (Muth6n, 1985; 

MuthSn & Kaplan, 1985)* Muth6n's (1985) TOBIT approach is a two- 

* 

stage estimation of s , where the initial stage is the univariate 

2 

estimation of y and a based on a normal distribution for 
censored variables (Gupta, 1952)* The second stage estimates the 
covariances by maximum likelihood method from the bivariate 
information by holding jjl and a constant at the univariate 
estimated levels (Muth6n, 1985)* This approach may be visually 
conceptualized in figure 3, where a latent correlation is 
estimated for two censored variables* 



Insert Figure 3 about here 



Results 



The results of Monte Carlo study number 1 support previous 
findings (Anderson & Gerbing, 1984; Boomsma, 1985) that 
nonconvergence rate is inversely related to sample size at 0% 
censoring. Figure 4a and 4b show the frequency of nonconvergence 
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for the NTGLS estimates for the single factor model over two 
levels model reliability (loading sizes) , four levels of sample 
size and four levels of variable censoring. 



Insert Figures 4a and 4b about here 



The asymmetry provided in the. higher censoring conditions seems 
to enhance nonconvergence, but more in the larger loading models 
than the smaller, with the exception of the M =25 condition. 
Figures 5a and 5b indicate that variable censoring has a dramatic 
effect on the proportion of congeneric model rejections, though 
one must be cautioned about the interpretation of the proportion 
of model rejections for small samples size (e.g., N=25 and N=50). 



Insert Figures 5a and 5b about here 



For convenience, the amount of bias will be studied in only 
two of the four \ estimates ( ? Vl a noncensored continuous 
variable, and \^ a censored continuous variable). Figures 6a and 
present the percent of bias in these two estimates over the 
studied conditions. In the censored variable (X A ), the percent of 
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bias seems to be a monotonic function of the amount of variable 

censor, regardless of the sample size, with slightly stronger 

bias occurring in the highly reliable model (large loading size). 

If one disregards the L 2 test on the basis of smaller sample 

sizes and calculates individual rater reliability, since 

2 

individual reliability may be reduced to \ i in this example, we 
may interpret Figure 6a as the amount of bias in the estimate of 
reliability for rater four. 

The amount of bias in the noncensored continuous variable 
may be considered trivial with a slight increase in model 
censoring. The bias in the higher reliability model ( K L = .9) 
tended to be negative, whereas the lower reliability model ( ^ = 
.6) tended to fluctuate more (Figure 6b). 



Insert Figures 6a and 6b about here 



A much more dramatic occurrence of negative bias may be seen 
in figure 7a, the percent of bias in the estimate of the error 
variance (6 g ) in the censored variable. Here one may see a strong 
negative bias even in the low censoring conditions. Sample size, 
in this example, seems to have little effect on bias once 
censoring has occurred. Model _ loading size produced a slight 
difference in bias, with the less reliable model providing 
stronger bias. Figure 7b show the percent of bias in the estimate 
of error variance in the non-censored variable. As seen in figure 
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7a, the less reliable model (smaller loadings) seems more 
susceptible to bias in the estimate of error variance. 



Insert Figures 7a and 7b about here 



Using the results from study number one, a second Monte 
Carlo study was designed using a single sample size (N-100) on 
the single congeneric model (see Figure 2). Figure 8 provides the 
proportion of model rejections for the three estimation 
procedures (NTGLS, ADF and TOBIT) across the four levels of 
variable censoring. 



Insert Figure 8 about here 



The ADF procedure was initially incorporated to deal with 
the non-normality of the censored models, and the TOBIT procedure 
was incorporated to deal with both non-normality and the 
attenuation in the correlation estimates. All three procedures 
similarly reject few congeneric models at zero censoring, but as 
one increases censoring the proportion of rejection increases in 
both NTGLS and ADF. NTGLS procedures in this example are not at 
all robust at the 50% censoring level, with the AL'F procedure 
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'also providing poor results* This may also be seen in figures 9a 
and 9b. 



Insert Figures 9a and 9b about here 



Figure 9a shows the percent of bias occurring in the 
loading estimate (one of the censored conditions) . Again the 
percent of bias is monotonically related to the amount of 
censoring in both NTGLS and ADF estimation procedures* The TOBIT 
estimates on the other hand, produced very little bias* The 
percent of bias occurring for \ x (noncensored condition) was 
negligible for the TOBIT and NTGLS estimates, but resulted in 
larger negative bias with the use of the ADF procedure (Figure 
9b). 

The percent of bias in the estimate of the error variance 
also resulted in a strong positive mono tonic relationship with 
the level of censoring for both NTGLS and ADF procedures* TOBIT 
estimates were consistently positively biased, but at a 
relatively low level (figure 10a)* Figure 10b shows the percent 
of bias in the estimate of the error variance for the non- 
censored condition* The most striking result in this figure is 
the consistent positive bias produced by the TOBIT estimates* 



Insert Figures 10a and 10b about here 
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Discussion 



Monte Carlo study number one demonstrated very well the lack 

of robustness in the NTGLS estimation procedure for retrieving 

correct \ parameter estimates, estimated error variances 9 8 f s, 

2 

and model fit indices (likelihood ratios (L )) at 25% or more 
censoring. One roust also be impressed with the general lack of 
model convergence with a small sample size (tf=>25) for such a well 
defined model. This point in itself may prove to be the limiting 
factor against the use of congeneric modeling techniques in 
interrater reliability studies. While not reported in this paper, 
concern for improper solutions (e*g. t . negative 8 g estimates) have 
also been shown to be related to small sample size, which would 
cause difficulty in estimating reliability. 

One may conclude from the first Monte Carlo study, that the 
use of Joreskog's (1970) congeneric modeling approach to 
interrater reliability estimation using HTGLS methods must be 
conducted very cautiously. While superior in many ways to the 
ANOVA approaches, the congeneric model is very susceptible to 
even mild deviations from normality (e.g., censored variables) 
and requires a moderately large sampling (at least N=100) for 
appropriate estimation. If one is not willing to accept these 
limitations, then one will certainly have to acc t limitations 
in the reliability estimates. 
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Monte Carlo study number two demonstrated that dealing with 
non-normal i ty alone ( e . g ♦ , ADF procedure ) is not enough for 
providing accurate reliability estimates in censored variable 
models* One must rlso be concerned with the attenuation in the 
correlations. This study demonstrated the superiority of the 
TOBIT estimates in dealing with these two conditions. If one can 
assume an underlying normality position, then latent correlation 
based estimates of reliability may prove to be the savior of the 
congener ic model ing approach to interrater rel iabil ity , though 
sample size i* still a limiting factor. While the latent 
correlation based estimates seem attractive, further Monte Carlo 
studies need to be conducted before they can be fully accepted. 
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Table 1 

Monte Carlo Study Design Number 1 



Estimation Number of Loading Sample Percent Number of 
procedure variables sizes sizes of censor replications 
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Two of the four X r s remained continuous (0% censor) throughout 
the study for internal comparison. 
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Table 2 

Monte Carlo Study Design Number 2 



Estimation Number of Loading Sample Percent Number of 
procedure variables sizes sizes of censor replications 
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Two of the four \ r s remained continuous (0% censor) throughout 
the study for internal comparison. 
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Table 3 

Model Population Values and Corresponding Standard Errors 



Standard Errors 



Parameter 


Value 


N=25 


N=50 


N=100 


N=400 


Large Loading 












*1 *2 *3 *4 


.900 


.160 


. 112 


.079 


.039 


e 6 6 6 

c c p e 
E l E 2 E 3 e 4 


.190 


.074 


.052 


.037 


.018 


Moderate Loading 












^1*2*3*4 


.600 


.232 


. 163 


.114 


.057 


6 e e 6 

£ 1 £ 2 £ 3 £ 4 


.640 


.250 


.175 


.123 


.061 




Ierjc 



Figure 2 
Single Factor Congeneric Model 
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Concept of a latent correlation 
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FIGURE 4a 

FREQUENCY €5? KCm^VSKGSKSS 

FREQUENCY' 







if* ' 


I 














1 




EL. w 








Wmm 





Q5S 25% 50% 75% 



PERCENT OF CENSORING 

E3 N-26 E3 N-50 EZ3 N-100 EZa N-400 

HIGH RELIABILITY MODEL Lambda - .9 
NTGLS ESTIMATION 



32 



FIGURE 4b 

FREQUENCY GF KCf^CNVEROENGS 
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FIGURE 5a 

FROPOSTOJ OF CC*H3EM£R3C MZVZL REJECTION 
HIGH RELIABILITY UHCEEL Lambda » £0 
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IFiGURE 5b 

PSCPCRTiCN GF CONGENERIC fcSCDZL REJECTION 
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FIGURE 6a 

F2RCSNT OF BIAS IN LAMBDA (4,1) ESTIMATE 
HIGH AND MODERATE RELIABILITY MODELS 
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FIGURE 6b 

PERCENT C? BIAS IN LAM3DA (1,1) ESTIMATE 
HIGH AND MODERATE RELIABILITY MODELS 
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FIGURE 7a 

PERCENT OF 28A3 IN THETA-EPS5LCN KD 
HIGH AND MODERATE RELIABILITY MODELS 
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FIGURE 7b 

PERCENT OF BIAS IN THETA-EP31.CN (1,1) 
HIGH AND MODERATE RELIABILITY MCDEL3 



20 
10 
0 
-10 
-20 
-30 
-40 



PERCENT OF ESTIMATE BIAS 



WW D WW m 



0" 



fl m..m 



MLM-0»MLM-26S5MLM-60«MLM-76a LLM-05S LLM-26*LLM-60»LLM-76» 

PERCENT OF CENSORING PER MODEL 

EZ3 N-25 CO N-60 ED N-100 EH N-400 

NTGLS ESTIMATION 

MLM - Moderate Loadings Mod9l 

LLM - Largs Loadings Modal 



FIGURE 6 

PRCPCRTtCN C? MODEL REJECT8CN3 
CCNTRCLUNQ FCR KCNCCNVERQENCS 
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FIGURE 0a 

PERCENT OF BIAS FOR LAMBDA (4,1) 
SAMPLE SIZE » 100 CVER 100 REPLICATIONS 
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FIGURE 9b 

PERCENT C? BIAS PCS* LAMBDA (1,1) 
SAMPLE SJ2E " 100 OVER 100 REPLICATIONS 
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FIGURE 10a 

PERCENT OF mm IN THETA-SP3HGN (4,1) 
££SOS3 THE THREE EBTI&S&TIGN PROCEDURES 
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FIGURE 10b 

PERCENT Cr BIAS THETa-EPSILOJi (1,1) 
#C?IOSS THE THREE ESTI&SOTSCN PROCEDURES 
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